Robust multi-modal medical image fusion via anisotropic heat diffusion guided low-rank structural analysis

Keywords: Multi-modal image fusion Data-specific filter Anisotropic heat kernel design Low-rank analysis Multi-scale decomposition a b s t r a c t This paper proposes a novel and robust multi-modal medical image fusion method, which is built upon a novel framework comprising multi-scale image decomposition based on anisotropic heat kernel design, scale-aware salient information extraction based on low-rank analysis, and scale-specific fusion rules. Our framework respects multi-scale structure features, while being robust to complex noise perturbation. First, anisotropic heat kernel is computed by constructing an image pyramid and embedding multi-level image properties into 2D manifolds in a divide-and-conquer way, consequently, multi-scale structure-preserving image decomposition can be accommodated. Second, to extract meaningfully scale-aware salient information, we conduct low-rank analysis over the image layer groups obtained in the first step, and employ the low-rank components to form the scale space of the salient features, wherein the underlying noise can be synchronously decoupled in a natural way. Third, to better fuse the complementary salient information extracted from multi-modal images, we design an S-shaped weighting function to fuse the large-scale layers, and employ the maximum selection principle to handle the small-scale layers. Moreover, we have conducted extensive experiments on MRI and PET/SPECT images. The comprehensive and quantitative comparisons with state-of-the-art methods demonstrate the informativeness, accuracy, robustness, and versatility of our novel approach. The objective of data fusion is to integrate the inherent complementary information from two or more resource images into a single one, which is more informative and suitable for human visual perception and further processing tasks [1]. In clinical medicine, magnetic resonance imaging (MRI) and computed tomography (CT) mainly reveal the structural and anatomical information, while positron emission tomography (PET) and single photon emission computed tomography (SPECT) concentrate on the functional information. Thus, multi-modal medical image fusion can provide much more anatomical and physiological characteristics, which has been playing a critical role in image analysis, clinical diagnosis, and treatment planning [2]. As an out-of-the-box tool in image processing and computational vision, recent progresses of Multi-scale Image Decomposition (MID) indicate that MID has promising potential in the improvement of image fusion quality, because it can effectively detach the structure-sensitive information from multi-modal images and re-integrate them to enhance informative perception via frequency-domain representation [3,4] or intensity domain presentation [5,6]. Meanwhile, since the design of MID kernel function has great influence on the fusion quality, to get the built-in attractive properties such as structure-sensitive, …